View Issue Details

IDProjectCategoryView StatusLast Update
0000615T99X171.00 SKB EagleSWpublic2021-12-23 17:23
Reporter(ALTech) SY Yoon Assigned To(SW) Jason LingDue Date2019-04-12 00:59
PriorityhighSeveritys2-severeReproducibilityhave not tried
Status closedResolutionfixed 
Summary0000615: Google Certification pretest
DescriptionSKB asked us to provide CTS/GTS Test result from current SW version v15.502.10.
Could you run CTS/GTS test and provide the result ?
TagsNo tags attached.
Attach Tags

Relationships

related to 0000769 closed(ALTech) Jason Park [Eagle][Common] CTS Test status 

Users monitoring this issue

User List (ALTech) Juyoung Park , (ALTech) SY Yoon , (SW) Jason Ling

Activities

(SW) Jason Ling

2019-04-10 09:48

developer   ~0001948

Mr Yoon,

We plan to run the CTS/GTS today, we think we could provide the reports this Friday.

(SW) Jason Ling

2019-04-12 16:59

developer   ~0001991

Hi Mr Yoon,

Please refer to attachment.

The test firmware is based on v15.502.10.
After first run, we use the latest source code to retry the remaining fail items.
cts.zip (1,497,646 bytes)
gts.zip (176,045 bytes)

(SW) Jason Ling

2019-04-18 10:38

developer   ~0002073

Hi Mr Yoon,

We found an issue from VTS (Vendor Test Suite) Certification pretest.
Please help us to provide the attachment to TVStorm.

TVS declared VINTF object "tvstorm.mmf" in device/amlogic/BFX-AT100/manifest.xml
The manifest.xml will be copied to vendor partition.
But TVS put the corresponding libraries into system partition.

Before VTS testing. We need to replace system partition with Google system image, so the corresponding libraries will be removed.

The test tool find tvstorm.mmf declaration which be defined in manifest.xml, but it can't find corresponding services.
vts.zip (27,480 bytes)

(SW) Jason Ling

2019-04-29 16:52

developer   ~0002280

Hi Mr Yoon,

Please check the attachment for CTS/GTS test result of Firmware V15.502.15.
CTS_GTS_V15.502.15.zip (1,689,627 bytes)

(ALTech) SY Yoon

2019-06-18 15:47

developer   ~0002713

Hi Mr. Jason Ling,

Could you provide test result with SW v15.502.39 ?
SKB asked us to check the Google test issues in our latest SW version.

(SW) Jason Ling

2019-06-18 16:06

developer   ~0002715

Hi Mr Yoon,

You can check the attachments.

GTS Report is Clean, you can see 6 fail items, but it is waived by Google.
CTS Report is not Clean. We have found some fail items with Player and interrupt the test.
In Foxconn side: We have received new library to verify these fail items from Amlogic.
In Amlogic side: They use the new library to do aging test.

After we verifying this new library. we will upload it to bitbuckt and rebuild the new SW image to you.
cts_v39.zip (1,964,604 bytes)
gts_v39.zip (178,794 bytes)

(SW) Jason Ling

2019-06-25 09:55

developer   ~0002811

Hi Mr Yoon,

We have finished the cts/gts of SW V15.502.40, please check the attachments.

GTS Report is Clean, you can see 6 fail items, but it is waived by Google.
CTS Report is not Clean, we have found side effect on AML's library, the fail item is testStagefright_bug_34360591.

We reported this fail item to amlogic last weekend and verify the new Patch now.
Amlogic also do aging test with this new Patch.

After we verifying this new patch. we will upload it to bitbuckt.
cts_v40.zip (1,636,943 bytes)
gts_v40.zip (178,050 bytes)

(SW) Jason Ling

2019-07-03 14:10

developer   ~0002822

Hi, Jason Park,

The attachments is xTS report of Version 43.
We have uploaded the V41 to APFE before. Google TAM has reviewed it.
xTS results of V43 is same as V41.
CTS.7z (1,805,397 bytes)
GTS.7z (173,404 bytes)

(SW) Jason Ling

2019-10-15 14:18

developer   ~0003098

Hi Jason Park,

I attach CTS/GTS reports of V56, I also add some comments for fail item.

CTS 3 fails
android.net.cts.ConnectivityManagerTest#testGetMultipathPreference (This item is related to Wifi, This item has been waived on v43)
android.security.cts.EncryptionTest#testEncryption (TV device do not need to encrypt, This item has been waived on v43)
android.security.cts.ListeningPortsTest#testNoRemotelyAccessibleListeningUdpPorts (This item has been waived on v43)

GTS 3 fails
com.google.android.security.gts.EncryptionTest#testEncryption (TV device do not need to encrypt, This item has been waived on v43)
com.google.android.settings.gts.MADAComplianceTest#testMADACompliance (This item has been waived by Google automatically)
com.google.android.installpackageswhitelist.gts.GtsInstallPackagesWhitelistDeviceTest#testInstallerPackagesAgainstWhitelist==> (We are requesting waiver for this item now)
CTS_V56.zip (1,946,633 bytes)
GTS_V56.zip (301,277 bytes)

(SW) Jason Ling

2019-10-22 09:51

developer   ~0003127

Hi Mr. Yoon and Mr. Jason Park

We got one fail item on GTS with the latest source code, please see the attachment.
About this item, we can pass it with AndroidTV standard firmware, so I have asked TVStorem before, they said they passed this item with ScreenOff.

We verified it yesterday with the latest source code, it fail again, we retry it more than 10 times with Screen Off, but it hard to pass.
Could you help us to check this issue with TVS.
2019.10.22_09.41.56.zip (29,998 bytes)

(SW) Jason Ling

2019-10-22 11:06

developer   ~0003128

Last edited: 2019-10-22 11:07

Hi Mr. Yoon and Mr. Jason Park,

I have found some clues of this issue, but I am not sure if it is right.

In following steps
1. when I got fail of this item, I go to settings > apps > B tv
2. click Clear Data and Clear Cache
3. Test it again with "Screen ON", we can pass it.

I am still verifying it, you can provide this information to TVS.
Thanks,

(ALTech) Jason Park

2019-10-22 12:53

developer   ~0003129

hi Jason
I received some feedback from TVS.
If you have some comment, please let me know it.

-------------------------------------------------------------------------------
The fail item you sent us is passed or failed item randomly from the previous version.
After do factory reset, it can pass the test by repeatedly testing for only this test.
-------------------------------------------------------------------------------

Thanks

(SW) Jason Ling

2019-10-24 10:43

developer   ~0003138

Hi Mr. Jason Park

I think maybe TVS don't know the root cause, because we have retried it many times.
so currently we use our test method to pass it.

(SW) Jason Ling

2019-10-28 08:46

developer   ~0003151

Hi Mr. Jason Park

I attach CTS/GTS reports of V57, The result of V57 is the same as v56.
CTS_v57.zip (1,938,215 bytes)
GTS_v57.zip (301,098 bytes)

(ALTech) Jason Park

2019-10-28 11:15

developer   ~0003156

hi Jason

Thanks for your update.
how about testFocusChangeMediaToGuidance ? you also passed with using your method or just passed at this time.

Thanks.

(SW) Jason Ling

2019-10-28 11:59

developer   ~0003159

Hi Mr. Jason Park

Just pass at that time.
If you want to find the root cause of this item. I think you need to check this issue with the owner of B tv.

(SW) Jason Ling

2019-11-08 08:54

developer   ~0003183

Hi Mr. Jason Park,

3PL had finished the test with FW v57 yesterday, there were no issues reported.
3PL has create one link about test result and assign to Google TAM, but we can not see the content of that link because of permission (only reporter and Google can see it)
if we want to track this, we must ask 3PL and tell them about that link, 3PL will update the content of link to us.

(ALTech) Jason Park

2019-12-03 14:22

developer   ~0003223

hi Jason.

we receive a google certification with v57 for BFX-AT100.
If we step with new FW for CTS Certification, we will create new Mantis ID.

Thanks.

(ALTech) Jason Park

2019-12-27 11:15

developer   ~0003258

hi Jason.

as we talked in webchat, you already inform to Google TAM at 12/26.

Thanks.

(SW) Jason Ling

2020-01-02 09:40

developer   ~0003259

Hi Mr. Jason Park

I attach CTS/GTS reports of V60
CTS_v60.zip (1,642,709 bytes)
GTS_v60.zip (301,247 bytes)

(ALTech) Jason Park

2020-01-31 14:28

developer   ~0003301

hi Jason.

after finish internal test with v15.513.10, please update it.
if possible, please update a schedule of Google CTS after checking it with 3pl, google.

Thanks

(SW) Jason Ling

2020-02-12 09:21

developer   ~0003340

Hi Mr. Jason Park,

Google has approved this build (v15.513.10).
GTS_v15.513.10.zip (208,466 bytes)
CTS_v15.513.10.zip (1,946,726 bytes)

(ALTech) Jason Park

2020-02-17 11:32

developer   ~0003351

hi Jason.

Thanks for your support.
I will change a status of this mantis as resolve. if need to reopen, I will open it.

Thanks.

Issue History

Date Modified Username Field Change
2019-04-10 00:02 (ALTech) SY Yoon New Issue
2019-04-10 00:02 (ALTech) SY Yoon Status new => assigned
2019-04-10 00:02 (ALTech) SY Yoon Assigned To => (SW) Jacky Chiang
2019-04-10 00:02 (ALTech) SY Yoon Issue Monitored: (ALTech) Jewoo Lee
2019-04-10 00:02 (ALTech) SY Yoon Issue Monitored: (ALTech) Eric Kim
2019-04-10 00:03 (ALTech) SY Yoon Issue Monitored: (ALTech) Kian Choi
2019-04-10 09:25 (SW) Jacky Chiang Assigned To (SW) Jacky Chiang => (SW) Jason Ling
2019-04-10 09:48 (SW) Jason Ling Note Added: 0001948
2019-04-12 16:59 (SW) Jason Ling File Added: cts.zip
2019-04-12 16:59 (SW) Jason Ling File Added: gts.zip
2019-04-12 16:59 (SW) Jason Ling Note Added: 0001991
2019-04-12 16:59 (SW) Jason Ling Assigned To (SW) Jason Ling => (ALTech) SY Yoon
2019-04-18 10:38 (SW) Jason Ling File Added: vts.zip
2019-04-18 10:38 (SW) Jason Ling Note Added: 0002073
2019-04-29 16:52 (SW) Jason Ling File Added: CTS_GTS_V15.502.15.zip
2019-04-29 16:52 (SW) Jason Ling Note Added: 0002280
2019-04-29 16:55 (SW) Jason Ling Issue Monitored: (SW) Jason Ling
2019-06-18 15:43 (ALTech) SY Yoon Assigned To (ALTech) SY Yoon => (SW) Jason Ling
2019-06-18 15:47 (ALTech) SY Yoon Note Added: 0002713
2019-06-18 16:06 (SW) Jason Ling File Added: cts_v39.zip
2019-06-18 16:06 (SW) Jason Ling File Added: gts_v39.zip
2019-06-18 16:06 (SW) Jason Ling Note Added: 0002715
2019-06-25 09:55 (SW) Jason Ling File Added: cts_v40.zip
2019-06-25 09:55 (SW) Jason Ling File Added: gts_v40.zip
2019-06-25 09:55 (SW) Jason Ling File Added: testStagefright_bug_34360591.jpg
2019-06-25 09:55 (SW) Jason Ling Note Added: 0002811
2019-07-03 14:10 (SW) Jason Ling File Added: CTS.7z
2019-07-03 14:10 (SW) Jason Ling File Added: GTS.7z
2019-07-03 14:10 (SW) Jason Ling Note Added: 0002822
2019-09-27 14:58 (ALTech) Jason Park Relationship added related to 0000769
2019-10-15 14:18 (SW) Jason Ling File Added: CTS_V56.zip
2019-10-15 14:18 (SW) Jason Ling File Added: GTS_V56.zip
2019-10-15 14:18 (SW) Jason Ling Note Added: 0003098
2019-10-22 09:51 (SW) Jason Ling File Added: testFocusChangeMediaToGuidance.jpg
2019-10-22 09:51 (SW) Jason Ling File Added: 2019.10.22_09.41.56.zip
2019-10-22 09:51 (SW) Jason Ling Note Added: 0003127
2019-10-22 11:06 (SW) Jason Ling Note Added: 0003128
2019-10-22 11:07 (SW) Jason Ling Note Edited: 0003128
2019-10-22 12:53 (ALTech) Jason Park Note Added: 0003129
2019-10-24 10:43 (SW) Jason Ling Note Added: 0003138
2019-10-28 08:46 (SW) Jason Ling File Added: CTS_v57.zip
2019-10-28 08:46 (SW) Jason Ling File Added: GTS_v57.zip
2019-10-28 08:46 (SW) Jason Ling Note Added: 0003151
2019-10-28 11:15 (ALTech) Jason Park Note Added: 0003156
2019-10-28 11:59 (SW) Jason Ling Note Added: 0003159
2019-11-08 08:54 (SW) Jason Ling Note Added: 0003183
2019-12-03 14:22 (ALTech) Jason Park Note Added: 0003223
2019-12-03 14:23 (ALTech) Jason Park Status assigned => resolved
2019-12-03 14:23 (ALTech) Jason Park Resolution open => fixed
2019-12-18 16:14 (ALTech) Jason Park Issue End Monitor: (ALTech) Jewoo Lee
2019-12-18 16:14 (ALTech) Jason Park Issue End Monitor: (ALTech) Eric Kim
2019-12-18 16:14 (ALTech) Jason Park Issue End Monitor: (ALTech) Kian Choi
2019-12-18 16:15 (ALTech) Jason Park Issue Monitored: (ALTech) SY Yoon
2019-12-18 16:15 (ALTech) Jason Park Issue Monitored: (ALTech) Juyoung Park
2019-12-18 16:18 (ALTech) Jason Park Status resolved => assigned
2019-12-27 11:15 (ALTech) Jason Park Note Added: 0003258
2020-01-02 09:40 (SW) Jason Ling File Added: CTS_v60.zip
2020-01-02 09:40 (SW) Jason Ling File Added: GTS_v60.zip
2020-01-02 09:40 (SW) Jason Ling Note Added: 0003259
2020-01-31 14:28 (ALTech) Jason Park Note Added: 0003301
2020-02-12 09:21 (SW) Jason Ling File Added: GTS_v15.513.10.zip
2020-02-12 09:21 (SW) Jason Ling File Added: CTS_v15.513.10.zip
2020-02-12 09:21 (SW) Jason Ling Note Added: 0003340
2020-02-17 11:32 (ALTech) Jason Park Note Added: 0003351
2020-02-17 11:32 (ALTech) Jason Park Status assigned => resolved
2021-12-23 17:23 (SW) Jacky Chiang Status resolved => closed